Construction of bilingual multimodal corpora of referring expressions in collaborative problem solving
نویسندگان
چکیده
This paper presents on-going work on constructing bilingual multimodal corpora of referring expressions in collaborative problem solving for English and Japanese. The corpora were collected from dialogues in which two participants collaboratively solved Tangram puzzles with a puzzle simulator. Extra-linguistic information such as operations on puzzle pieces, mouse cursor position and piece positions were recorded in synchronisation with utterances. The speech data was transcribed and time-aligned with the extra-linguistic information. Referring expressions in utterances that refer to puzzle pieces were annotated in terms of their spans, their referents and their other attributes. The Japanese corpus has already been completed, but the English counterpart is still undergoing annotation. We have conducted a preliminary comparative analysis of both corpora, mainly with respect to task completion time, task success rates and attributes of referring expressions. These corpora showed significant differences in task completion time and success rate.
منابع مشابه
The REX corpora: A collection of multimodal corpora of referring expressions in collaborative problem solving dialogues
This paper describes a collection of multimodal corpora of referring expressions, the REX corpora. The corpora have two notable features, namely (1) they include time-aligned extra-linguistic information such as participant actions and eye-gaze on top of linguistic information, (2) dialogues were collected with various configurations in terms of the puzzle type, hinting and language. After desc...
متن کاملTowards a Balanced Corpus of Multimodal Referring Expressions in Dialogue
This paper describes an experiment in which dialogues are elicited through an identification task. Currently we are transcribing the collected data. The primary purpose of the experiment is to test a number of hypotheses regarding both the production and perception of multimodal referring expressions. To achieve this, the experiment was designed such that a number of factors (prior reference, f...
متن کاملSOLVING BEST PATH PROBLEM ON MULTIMODAL TRANSPORTATION NETWORKS WITH FUZZY COSTS
Numerous algorithms have been proposed to solve the shortest-pathproblem; many of them consider a single-mode network and crispcosts. Other attempts have addressed the problem of fuzzy costs ina single-mode network, the so-called fuzzy shortest-path problem(FSPP). The main contribution of the present work is to solve theoptimum path problem in a multimodal transportation network, inwhich the co...
متن کاملREX-J: Japanese referring expression corpus of situated dialogs
Identifying objects in conversation is a fundamental human capability necessary to achieve efficient collaboration on any real world task. Hence the deepening of our understanding of human referential behaviour is indispensable for the creation of systems that collaborate with humans in a meaningful way. We present the construction of REX-J, a multi-modal Japanese corpus of referring expression...
متن کاملA model for distribution centers location-routing problem on a multimodal transportation network with a meta-heuristic solving approach
Nowadays, organizations have to compete with different competitors in regional, national and international levels, so they have to improve their competition capabilities to survive against competitors. Undertaking activities on a global scale requires a proper distribution system which could take advantages of different transportation modes. Accordingly, the present paper addresses a location-r...
متن کامل